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ABSTRACT 


Developing innovative science, technology, engineering and mathematics (STEM) curricula that 
elicit student excitement for learning is a continuous challenge for K-12 STEM teachers. Generating 
these lessons while meeting conflicting pedagogical objectives and constraints of time, content, and 
cost from various parties is truly a challenging task for any teacher. Recognizing the parallel between 
curricular design and engineering design, we posit that the engineering design process (EDP) can 
be used as an innovative, effective, and logical means for formalizing the development of K-12 STEM 
lessons. The use of the EDP as an instructional development (ID) model is firmly based on the existing 
literature theory of how people learn; in particular, identifying the students and teachers as clients for 
the design process is a learner-centered approach, and problem-based learning (PBL) encourages ac¬ 
tive learning of STEM concepts in the context of authentic problems. To develop this process in practice, 
we collaborated with 15 middle school teachers over three years to create active learning curricular 
modules to teach over 2,000 students difficult STEM concepts. This article describes the practice of 
how teachers can utilize the EDP to develop problem-based curricular units and in doing so become 
more comfortable with the EDP themselves. We also report on the evaluation of this project. 

Key Words: Curriculum development, K-12 outreach, Middle School, Active learning, Engineering 
design process, Evaluation 
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INTRODUCTION 


Our society is increasingly dependent on science, technology, engineering, and mathematics 
(STEM) knowledge, yet students who fail to take preparatory STEM courses limit their future abili¬ 
ties to pursue technological careers. The percentage of top American students entering science and 
engineering fields has declined in recent years, and underrepresented minorities and women are 
disproportionately absent in STEM fields (NSB National Science Board May 5, 2010). It is becom¬ 
ing clear that not enough students are pursuing STEM to meet the nation’s technological needs 
(Rockland, et al. 2010, Douglas, Iversen and Kalyandurg November 2004). To counteract the lack 
of interest in STEM by many students, there is a pressing need for innovative STEM curricula that 
elicit student excitement for learning (Rockland, et al. 2010). 

Recent efforts to improve STEM education, and education in general, launched by national, state, 
and local policymakers, have created a fundamental shift in what is expected of teachers. New high 
standards of performance adopted by states and districts require new educational paradigms and 
professional development for teachers (Garet, et al. 2001). In response, many K-12 STEM teachers 
strive to develop interactive and engaging STEM programs to provide their students with deeper 
understanding of subject matter and problem-solving skills. In our particular case, middle school 
teachers who participate in our NSF-funded Research Experiences for Teachers (RET) program are 
required to design a lesson plan incorporating engineering design to teach an aspect of their STEM 
curriculum. Generating these lessons, while meeting conflicting pedagogical objectives, such as 
preparation for standardized tests versus mentored exploration, is truly a difficult task especially 
considering the constraints of time, content, and cost from various parties. 

While all grades benefit from STEM instruction, middle school is a crucial intervention point for encour¬ 
aging students to pursue engineering. Students report that math becomes more difficult, they receive less 
help from teachers and parents, and they become more anxious about technical material in the middle- 
school years (Adelman 1998). Thus, there is a critical need at all grade levels, and in middle school in 
particular, to provide a structured process to aid STEM teachers in the development of lesson plans with 
confidence that the activities will be innovative, relevant, and within the constraints of the teacher and 
classroom environment. We suggest that the engineering design process (EDP) is a logical way to provide 
a framework and context for developing STEM lessons. By its problem-solving nature, the EDP addresses 
conflicting objectives and constraints of the classroom. The EDP (specifically the steps enumerated in the 
Massachusetts’ Curriculum Frameworks in Figure 1) provides a mechanism for teachers to design engaging 
STEM lessons and at the same time aids them in developing competence and confidence in their teaching 
of engineering design. The reinforcement of teachers’ familiarity with the EDP itself is a major benefit of 
using the EDP for lesson development compared to other Instructional Design (ID) models. 
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This article describes the practice of middle school teachers using the engineering design process 
to develop problem-based curricular units, lessons, or activities (generally referred to as “units” 
herein) and in doing so become more comfortable with and competent in the EDP themselves. We 
begin with background information about curriculum development and professional development. 
Next, we describe the EDP and its adaptation to the design of STEM lessons, including an example 
from a middle-school teacher. We follow with our evaluation plan and findings. Finally, we discuss 
our conclusions and implications for future use and dissemination. 


BACKGROUND 

Low student interest and enrollment in STEM studies is a matter of national and international concern. 
Middle school is an optimal time for educational intervention towards increasing participation (Carnegie 
Council on Adolescent Development 1995), as children’s science ambitions are largely formed in the ages 
between 10 to 14 (Archer, et al. 2012). Pedagogical research indicates that the best way to engage middle 
school students is to center their educational activities on active, engaged thinking and inquiry, and on 
developing critical thinking skills (Ambrose, et al. 2010, Bransford, Brown and Cocking 2000, Duschel 2007, 
Jonassen 2000, Penuel, et al. 2007). A range of instructional approaches which includes student-directed 
activities, practice, experimentation, along with content knowledge is recommended (Duschel 2007). 

While middle school teachers are adept at offering these types of activities, their confidence in and 
knowledge of engineering may be limited (Robinson 1999). A National Academy of Sciences study showed 
that science teachers lack a firm understanding of engineering practices, uses, and concepts (Sciences 
1998). Among middle school teachers in the Schools and Staffing Survey, 51.5% of those who taught math 
and 40.0% of those who taught science did not have a major or minor in these subjects (Kuenzi 2008). 
Middle school science teachers also have fewer years teaching experience in their subject area than other 
teachers (Braun, et al. 2009). Further, compared to other STEM fields, K-12 engineering education is still in 
its infancy, even called “almost invisible,” in the United States (Katehi, Pearson and Feder 2009). In Mas¬ 
sachusetts, the first state to implement K-12 engineering standards, science teachers are increasingly asked 
to include engineering design in their classes. Yet, there is a dearth of teacher professional development, 
certification requirements, and pre-service preparation programs to help science teachers incorporate 
engineering. In a recent study of six Massachusetts middle school teachers, all six received a “high” rat¬ 
ing for their explanations of the “Construct a prototype” step in the EDP, yet none received this level for 
“Develop possible solution” and “Select best possible solution” (Hynes 2012). The limited ability to explain 
the critical early steps of the EDP is concerning considering the new curriculum standards’ emphasis on 
integration across disciplines which will increase all teachers’ need for familiarity with the EDP. 
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Teacher professional development is one means to address both design of engaging lesson plans and 
confidence with the EDP. An effective approach should provide peer support, teacher-developed research 
experiences, and hands-on curriculum development. Experience from other K-12 programs has shown that 
teachers will make best use of professional development activities that are content specific (Schaefer-Zarkske 
2004), occur over time, and are rooted in their classroom experiences (Atkin 2001, Opfer and Pedder 2011). 
These activities should also be inquiry-based and learning-centered (Anderson and Herr 2011). However, 
most professional development for teachers is not learner, knowledge, assessment, and community-centered 
(Bransford, Brown and Cocking 2000). We propose using the EDP itself as a learner-centered model of 
Instructional Design (ID) to develop problem-based EDP curricular lessons or units at the K-12 level. 

An EDP approach to curriculum development is within the tradition of teacher inquiry and professional 
development theory. Not only does it provide the opportunity for teachers to go through the same pro¬ 
cess that their students will, it also provides an intellectual and organizational mechanism for teachers to 
create and implement new lessons. When teaching and developing lessons, teachers continually assess 
the task, plan an effective approach, monitor progress, evaluate and adjust their approach or materials 
(Ambrose, et al. 2010). Although many teachers already design, research, and interpret their teaching 
(Dana and Yendel-Hoppey 2009), the EDP provides a framework and tools for these activities. 

The EDP approach is similar to other models of Instructional Design (a.k.a., Instructional Systems 
Design), yet it has unique differences. ID is a generic term for systematic analysis and development of 
learning needs and instruction and involves many models. Virtually all ID models include analysis, de¬ 
sign, production, and evaluation steps, which are strikingly similar to the steps in the EDP (see Figure 1) 
(Gustafson and Branch 1997). For example, a common expression of ID is in the ADDIE model, an acronym 
referring to Analysis, Design, Development, Implementation, and Evaluation (Molenda 2003). Another 
is the ASSURE model, an acronym referring to Analyze learners, State objectives, Select media and ma¬ 
terials, Utilize materials, Require learner participation, Evaluation/review (Gustafson and Branch 1997). 
Missing from these approaches, however, is the EDP emphasis on creating various approaches (multiple 
alternative designs) from which the best is selected by careful (and often quantitative) weighing against 
curriculum objectives and implementation constraints. The EDP also provides a tool for conceptual¬ 
izing and communicating which, consistent with the goals of all ID models according to Gustafson and 
Branch (1997), aid in visualizing, directing and managing the creation of “episodes of guided learning”. 
Finally, revision is not an explicit step in many ID models (e.g., it is missing from the ADDIE and ASSURE 
acronyms), whereas it is unequivocally present in the EDP. Whereas ID can be depicted by rectilinear or 
curvilinear process charts (Gustafson and Branch 1997) the EDP is often represented as a cycle, generally 
depicted by stages in a circular path (e.g., Figure 1), indicating a continual process of revision. 

While the EDP approach to curriculum intervention that we describe here was implemented as a 
collaboration between middle school teachers and researchers, it is a tool for individual teachers at 
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Figure 1. The steps of the Engineering Design Process (EDP) based on the Massachusetts’ 
Curriculum Frameworks. A few feedback loops are indicated by arrows within the circle, 
although additional sub-loops can occur between any two steps. 


all grade levels. Our focus is on practice and developing a particular lesson. Further, collaboration 
with a researcher could enhance the curriculum development, but it is not required. It is expected 
that the EDP will be most useful for design of individual projects (units, lessons, or activities) on 
relatively constrained concepts or topics that teachers find their students having difficultly learning, 
e.g., the interconnection between density, weight, mass, and gravity or why poor diet can lead to 
clogged arteries and reduced blood flow, rather than development of an entire STEM curriculum. 


APPLYING THE ENGINEERING DESIGN PROCESS (EDP) TO LESSON DEVELOPMENT 


While there are many different models to express the EDP, they follow a similar iterative and 
systematic process that focuses on client needs, research, idea generation, ranking objectives and 
choosing the best possible solution or solutions to meet specifications and constraints. Solutions 
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are then implemented, communicated, evaluated, and redesigned. This approach is outlined in the 
National Research Council’s Framework for K-12 Science Education (Standards 2011) and in engi¬ 
neering design textbooks (Dym and Little 2009). The EDP differs from scientific inquiry in that it is 
cyclic rather than linear because the goal of the process is to obtain a useful solution to a problem 
rather than to test a hypothesis, and generating a solution often requires iteration. 

The steps of the EDP are presented in Figure 1. This representation is similar to that outlined in 
Massachusetts’ Curriculum Frameworks (Education 2006) for teaching engineering design. Although 
the numbers of steps, terminology for the steps, and interdependence shown between the steps 
varies between representations (e.g., some textbooks include communication explicitly (Dym and 
Little 2009) and others include funding approval as a separate step (Ertas and Jones, The Engineer¬ 
ing Design Process 1996), all descriptions of the EDP demonstrate the cyclic nature of the process 
including opportunities for feedback, and all focus on clients’ needs, objectives, and constraints. 

When a general problem or need is recognized, it must be clarified and revised based on background 
research, interviews with stakeholders, prioritizing objectives, and determining constraints. Multiple 
solutions that address goals, objectives, and constraints are then generated with the help of various 
design tools (e.g., brainstorming, comparison charts, etc.). The designers are then able to choose the 
best of the alternative designs based on how well each potentially meets the objectives. Construction 
of a prototype or model and its testing and evaluation demonstrate proof-of-concept and degree of 
functionality. The results of testing/evaluation are presented to peers formally and informally to docu¬ 
ment the product and for feedback. Reassessment and review follow, with redesign as needed. 

These steps of identifying, approaching and solving engineering design problems have analogous 
meanings in the development of curricula. With a focus on “client” (student) needs and ranking of 
objectives, while working within constraints, the EDP provides a systematic framework for development 
of effective classroom instruction. In Table 1, the adaptation of the EDP for the process of curriculum 
design is displayed. As with the EDP shown in Figure 1, the application of the EDP to curriculum design 
is also cyclic in nature in terms of the entire process and also between intermediate steps. 


IMPLEMENTATION 


This EDP approach to curriculum design was piloted in an NSF RET program at Worcester 
Polytechnic Institute that provides opportunities for teachers to practice research in collaboration 
with college faculty. Funded over a three-year period, 15 middle-school teachers received in-depth 
exposure to one area of biomedical engineering through a six-week, hands-on, inquiry-based research 
laboratory experience involving bioengineering design. Teachers also gained breadth of knowledge 
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Engineering Design Process 

STEM Curriculum Design Process 

Identify the Problem/Need 

Identify difficult topics for students to learn and specify educational objectives or 
learning expectations. 

Research and Rank Objectives 
and Constraints 

Research technical and pedagogical approaches to teaching the material; consider context 
and constraints to teaching; rank learning objectives or expectations and constraints to 
implementation. 

Develop Possible Solutions 

Come up with multiple curriculum design ideas that could be used to teach the material. 

Select the Best Solution within 
Constraints 

Choose the idea that best meets educational objectives and classroom context and 
constraints. 

Construct a Prototype!Model 
Solution 

Develop an initial curricular unit. 

Test/Evaluate the Solution 

Test the curricular unit in their classrooms during the school year. 

Present!Communicate the 
Results 

Present and discuss findings with peers, administrators, and others to obtain feedback 
and suggestions. 

Redesign and Revise 

Begin the redesign process based on assessment and on feedback. 

Table 1. Adapting the steps of the Engineering Design Process for K-12 STEM curricular development 


by exposure to research projects of other RET participants and by attending professional work¬ 
shops. The teachers’ research experiences served as a catalyst for their curriculum development. 
Following the EDP, the teachers developed bioengineering design projects for their classrooms that 
would both teach difficult STEM concepts and engage their students’ learning and motivation. The 
development of a curricular unit or lesson and the teaching of it the academic year following the 
summer program were required. 

The teachers taught sciences (66.7%), technology/engineering (13.3%), and special education 
(6.7%) to 6 th through 8 th graders. Most of the teachers were female (73.3%). Five teachers repeated 
the program once and one teacher enrolled for all three years of the grant. Over a three-year period, 
teachers taught their units to over 2,000 middle-school students. 

Effective professional development includes peer support, collaboration, and classroom application 
(Atkin 2001). To facilitate development of their lessons or units, teachers attended weekly, three-hour 
discussion and planning meetings in which they presented their progress through the EDP. Each teacher 
spoke for a few minutes and then answered questions and received feedback from the group. Facili¬ 
tated by an external evaluator, the sessions were teacher-driven. Although program directors were 
occasionally invited to attend to provide feedback on technical topics and to clarify directions, teachers 
primarily worked on their own in the meetings. During the meetings, the teachers were supportive of 
one another, even while critiquing presentations. In particular, more experienced teachers and those 
who were in the program for more than one year helped newer and less experienced teachers. 
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With their middle school students identified as the clients, teachers began by identifying a prob¬ 
lem or a learning need, i.e., a concept that they find difficult to teach or for their students to grasp. 
Teachers then researched content and teaching methods related to the problem and presented dif¬ 
ferent ways to address it. While most teachers started with different problems and asked for help 
in determining which one to use, other teachers presented just one problem and different ways to 
address it in the classroom. The difference between deciding on a topic and presenting ways to ad¬ 
dress or teach it was problematic for some teachers. Because some teachers have a clearer idea of 
students’ needs than others, it is vital to have teachers express what learning they (and their state’s 
standards) consider important before deciding on solutions or ways to present the content. 

After some discussion on the differences between objectives (what one hopes to have happen, 
planned activities and content) and learning outcomes (what knowledge, attitudes students are 
expected to have or skills to demonstrate), teachers articulated curriculum objectives and learning 
outcomes. As teachers researched, considered constraints, and presented ideas, they often modi¬ 
fied their objectives and outcomes. 

One of the first constraints that teachers identified was the need to align a proposed topic with the 
standards in the Massachusetts Curriculum Frameworks and their local school districts’ curricula. With most 
teachers required to post related framework or standards in their rooms daily, this is a considerable constraint. 
Teachers offered suggestions to each other on how a topic might fit within frameworks and gave examples 
of how teachers might create or present a topic. Other constraints such as time, resources, and students’ 
grade level were clearly enumerated. For inner-city teachers, in particular, resources are an issue. 

After the objectives were ranked in order of importance, the constraints set, and the outcomes 
delineated, the teachers revised their original problem statement so that their goals and deliverables 
were clear and their progress towards those goals could be assessed. The teachers were then asked 
to develop multiple “design alternatives,” i.e., curricular units or lessons that could potentially be used 
to meet their educational objectives. These alternatives were then weighed against the objectives 
and constraints to determine the optimal solution. While various approaches were discussed during 
group sessions and with program faculty leaders, the final choice of a curricular unit or lesson was 
made by the individual teacher. 

Teachers developed the chosen unit as a “prototype” and presented it to their colleagues for 
feedback during the final week of the summer and made additional revisions as necessary. During the 
academic year following the summer of their participation, teachers taught their prototype lessons 
in their middle school science and technology classes. At follow-up sessions held on our institution’s 
campus, teachers presented their experiences teaching and assessing their curricula. 

Every teacher taught a curricular unit. While most teachers taught their lessons or activities in their 
regular classrooms, not all teachers were able to teach them within a structured course. Two teachers 
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received permission to create new courses based on the EDP and biomedical engineering. Three teach¬ 
ers taught their units as an afterschool activity. Four teachers reorganized their curriculum to include 
the units. They received support from school administrators and parent organizations to teach their 
units. For example, when one teacher’s teaching assignment changed over the summer to a different 
subject and grade, the administration gave the teacher permission to teach the unit as a review. 

The assessment plans that teachers created for their lessons or units were formative and summa- 
tive in nature. All teachers planned informal, formative assessment of students’ understanding and 
prior knowledge. Three teachers also used structured formative assessments that require students 
to answer two or three questions related to key concepts covered (e.g., exit slip) or to be covered 
during the class (e.g., bell work). Another structured formative assessment used by a teacher is 
a KWL chart in which students fill in columns labeled what I know (K), what I want to know (W), 
and what I learned (L). Teachers planned to employ rubrics (with varying level of detail) to assess 
prototypes and use of the EDP in creation of prototypes. Other types of assessments included lab 
notebooks, journals, brochures, posters, class presentations, worksheets, videos, and TV commer¬ 
cials. Five teachers surveyed their students’ attitudes and interest. Twelve teachers planned pre and 
post assessments of knowledge with quizzes and unit tests. 

During the summer, teachers also created a poster of the development of their “prototype” curriculum 
design to display during the final day of the summer program when they made a public oral presentation 
of their research projects. Teachers took the posters back to their schools to hang in their classrooms 
and halls, to share with colleagues, and to present to the school district and parents. See Figure 2 for the 
template teachers used to create a poster. In addition, teachers submitted their curriculum designs to Teach- 
Engineering.org after they were tested in the classroom with students and redesigned as necessary. 


ONE TEACHER’S APPLICATION OF THE EDP CURRICULUM PROCESS 


To show how the process was used, we describe the development of a curricular unit by one of the 
authors (Tom O.), a technology teacher of 7 th and 8 th graders in a small, low-income city school. First, 
Tom identified a general area of instruction that he wished to address. Because his students had limited 
experience with problem-based instruction and the EDP, he decided this would be his overall focus, 
with biomedical engineering concepts as an underlying motivational theme. Curriculum objectives 
were to increase student understanding of the EDP, student knowledge of biomedical engineering, and 
student problem-solving skills. Next, he considered curriculum constraints: students’ lack of experience 
working with open-ended problems and their limited background knowledge. Limited resources and 
time, and the need to meet specific state and local curriculum standards were also constraints. 
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MWPI 


Main title: 

Subtitle 

Your name 

WPI-NSF RET Program in Bioengineering, Worcester Polytechnic Institute, Worcester, MA 



Introduction 


Add bulleted text here with references - but keep 
text to a minimum 

Purpose of poster (design of curriculum unit), 
relationship to your research here at WPI, general 
description 

Make this something you're proud to show off for 
your students, their parents, school administrators, 
etc. 


Teaching Objectives and Constraints 


Short general statement of 
problem 

Bulleted list of objectives and 
constraints 
Topics covered 
Student grade 
Numbers of students 
Number and duration of class 
periods 
Etc. 


Research/Background 


Bulleted list of background 
information with citations 

Literature, patent searches, 
interviews 

Include alternative designs 


Chosen Solution 


Description of EDP problem for unit/lesson 

Bulleted lists describing your best solution 
Specific learning outcomes 
Project-specific constraints 
Plan for how you will implement it 


Add pictures (cite the source) 


Assessment 


Preliminary testing your solution against your 
objectives - summative assessment 

How you will document implementation of your 
unit and students' reactions - formative 
assessment 


Conclusions and Future 


Bulleted list of conclusions with citations to the 
literature 

What you learned form the process of creating a 
unit based on the EDP 

Where and when will unit be implemented, shared, 
revisions planned, etc. 
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Figure 2. Template for the EDP curriculum design poster. 


Tom generated four design alternatives and presented them to his summer colleagues during a 
weekly meeting. These were for the students to design 1) protective equipment for a sport, 2) exercise 
equipment to use in space, 3) a device to test tensile strength of potential tendon/ligament graft 
materials, and 4) a tissue substitute for anterior cruciate ligament (ACL) reconstruction in damaged 
knees. After considering the extent to which each would satisfy his objectives, the complexity, resources 
needed, appropriateness for his students, and the relationship to his research of each alternative, he 
decided on the fourth idea. The unit addressed a real problem with a clear biomedical engineering 
focus. Because a popular quarterback in the New England Patriots recently suffered a torn ACL, the 
problem was well known and of interest to students. Tom created a plan for an EDP for his 8 th grade 
technology students. At weekly Discussion Group meetings, he presented his progress. 

During the meetings, his peers gave him valuable feedback. For instance, they suggested that he 
articulate more specific objectives for the students, identify the parts of the EDP where his students 
have the most trouble, and offer his students a demonstration of the process. The group discussed 
how he and his students would know they were successful. 
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Toward the end of the summer, Tom presented his prototype curriculum design. He articulated the goal 
or problem statement as to design an ACL substitute to replace a torn ligament. Creating specific learning 
expectations, he considered the content and skills students would need. Students were to identify four 
examples of ligaments in the human body, apply the EDP to solve the problem, and communicate the 
results of their solution. Because of time limits and that the course was technology/engineering based, 
not biology or other science, constraints were specified with emphasis on the mechanical properties of 
the graft e.g., biocompatibility was not considered. Several physical constraints were specified for the 
students: minimum of 4" long, no larger than %" in diameter, must hold a minimum of 135 pounds, must 
maintain a stiffness of 10-40 Ibs/inch, and must be completed in two weeks. 

In the academic year following the summer program, Tom taught his unit to his Technology classes. 
He used the problem statement he developed over the summer: to design an ACL substitute to replace 
a torn ligament. The learning outcomes for his students were that students would be able to 

• identify three examples of tissues in the human body 

• apply the EDP to solve a problem, and 

• communicate the results of a design solution. 

After instruction on the EDP and the introduction of the problem statement, students worked in 
teams throughout the project, beginning with the related research and refinement of the problem 
(Fig. 3a). They then developed multiple solutions on paper and with string, chose the best design, 
developed a physical prototype (Fig. 3b), and evaluated their solution. The evaluation of the model 
was accomplished through the use of a simple custom-made device whereby students were able 



(a) (b) (c) (d) 

Figure 3. Middle school students (a) researching ligament structure and function and 
(b) creating a prototype ligament replacement by braiding leather strips, (c) Evaluation of a 
prototype using (d) a low-cost custom mechanical test setup employing hanging weights. 
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to conduct the tensile test on their “ACL substitute” using hanging weights and rulers and record 
data (Fig. 3c,d). Collected data were then input into a spreadsheet program to plot the results 
and calculate stiffness of their prototype (slope of the force/displacement plot). Due to time con¬ 
straints, this project did not address redesign, although preliminary proof-of-concept models were 
created before the final prototype to be tested. For a multimedia slideshow please see Movie SI 
[https://www.youtube.com/watch7v=jdJS8xs_3mg&list=PL84VcUp5xaBsRutUS-8KR2lwFxh8Kfl9G&index=4]. 
Tom also developed a separate curricular unit using the EDP where his students designed assis¬ 
tive devices to aid a disabled classmate in opening of cans and bottles. The students created and 
tested prototypes using tools and machines such as a band saw and drill press (see Movie S2 
[https://www.youtube.com/watch?v=izcKivgfHrk&list=PL84VcUp5xaBsRutUS-8KR2lwFxh8Kfl9G&index=5]. 

Tom used daily exit slips, a formative assessment, to spot check his students’ understanding of mate¬ 
rial covered throughout all class projects. Typical questions include: “What is the third step of the design 
process?” and “What is the purpose of a prototype?” In addition, he employed a summative assessment 
of the project. Creating and following a rubric uniquely developed for each project, he assessed students’ 
work on multiple components or deliverables. This assessment included parts for research, design de¬ 
velopment, prototype/model development, prototype assessment and communication and redesign. 


EVALUATION 


The program in which the EDP model for curriculum development was embedded was funded by 
an NSF development grant. For NSF’s evaluation requirements we articulated a series of evaluation 
questions related to all the program’s objectives. These questions covered formative and summative 
aspects of evaluation. In this paper, we focused only on those questions that related to the use of 
the EDP to develop a curricular unit. These questions were: 

• How many teachers developed and implemented an engineering design lesson unit? 

• To what extent do teachers indicate a change in their competency in the engineering design process? 

• To what extent have teachers’ self-ratings of competency to convey excitement about bioen¬ 
gineering to their students increased? 

• To what extent did teachers’ involvement in RET influence the content of what they teach and 
the strategies they use to teach or way they teach? 

Evaluation of the RET program and its EDP curriculum development component was ongoing. 
During the program, we used formative evaluation findings to make changes to the program and 
teachers’ curriculum development requirements. During the academic year, we used the data to 
plan for the next summer. 
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Several tools were used to gather information to answer our questions . In addition to conducting 
Discussion Groups and observations during the summer, we surveyed the teachers at the beginning 
and end of the program. 

Development of the survey was a multi-step process. First, we articulated our objectives and 
what we and other stakeholders needed to know. Next, we researched published and available sur¬ 
veys, but did not find one that was valid for our use. Either the content of the survey did not match 
the content and objectives of our program or the surveys were poorly constructed. Questions on 
our survey were then checked for alignment with program objectives and pilot-tested with a few 
middle-school teachers from another program. 

Analysis of the data included percentage of responses, means, and standard deviations. Because of 
the small sample size, we used non-parametric statistics to assess effects from the beginning to the 
end of the summer program. Patterns and trends were gleaned from content analysis of notes and 
transcripts from discussion groups, observations, and written comments on surveys. These data were 
triangulated with teachers’ ratings on surveys. The qualitative and quantitative data helped us make 
decisions about both the implementation and the effects of the program and EDP component. 

It should be noted that use of the EDP to create a lesson or curricular unit was developed during 
the first summer of the program when formative evaluation findings suggested that teachers needed a 
structure to help them create a lesson. Because using the EDP to develop a curricular unit was a work 
in progress, teachers, particularly in the first two years, had trouble with the changing expectations 
for the units. Yet, even with ongoing programmatic changes, teachers rated the experience highly. 
Each teacher developed a poster that displayed their use of the EDP in creating a curricular unit, and 
almost all of them proudly displayed the posters in their classrooms and during parent nights. 

Fifteen teachers participated, with five teachers coming for two summers and one teacher en¬ 
rolling for three summers. In year one, six teachers participated, while in years two and three, eight 
and seven teachers, respectively, participated. All teachers reported that they participated in the 
program “to broaden abilities by learning a new area.” Thirteen of the 15 reported participation “to 
learn how to teach engineering more effectively.” 

Over three-fourths (76.2%) of the participants rated the curriculum development component as 
“excellent” and 19% rated it as “good” (Table 2). The one teacher (4.8%) who rated it as “fair” noted that 
“traditional Instructional System Design Methods or style could be used for objectives. We seem to force 
the constraints or outcomes in objectives.” Several teachers wrote that they “enjoyed using the EDP to 
plan the curriculum" and that they plan to use it in the future and to share it with their colleagues. One 
teacher explained why the EDP curriculum development was helpful: 7 think it helped me focus the 
way / develop lessons and has given me a framework/structure for adhering [sic] my usual stream of 
consciousness ideas. It has helped me become more thoughtful and conscious of how i plan and what 
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Teachers’ Ratings 
by Year of Program 


Percentage of teachers 


Year 1 

(n = 6) 

Year 2 
(n = 8) 

Year 3 
(n = 7) 

Total 
(n = 21) 

Excellent 

100- 

75.0 

57.1 

76.2 

Good 

0 

12.5 

42.9 

19.0 

Fair 

0 

12.5 

0 

4.8 


a One teacher checked in between “Excellent” and “Good”. 


Table 2. Teachers’ ratings of the curriculum development process 


the purpose of the lesson is.” Although one teacher rated the curriculum development as “good," the 
teacher wrote 7 do not think i will use the EDP when planning curriculum.” The teacher did not provide 
reasons on the end-of-program survey. 

During their first year, all five repeating teachers rated the curriculum development as “excellent.” 
Just one of the five, changed the “excellent” rating to “good” after the second year of participation. It 
is not clear why there were fewer ratings of "excellent” in year three than in earlier years. (This trend 
did not continue in the first year of the next round of funding completed while this article was in 
review: 70% of the 10 participants rated curriculum development as “excellent”, 30% as “good.”) 

Although one of our evaluation questions is the extent to which the program influenced what and the 
way teachers teach, it is not directly related to the process of curriculum development. However, most 
(81.0%) of the teachers indicated that their participation “influenced the way they teach a great deal.” Given 
the constraints of state curriculum standards, the influence on content is considered good (Table 3). 

Teachers reported that their curriculum projects could help students learn not only the engineering 
design process itself (required in Massachusetts and other states), but also could facilitate teaching 
difficult STEM concepts such as density, temperature, and flow. Teachers remarked that they were 
“excited” about using the EDP in their classes. However, our Life Sciences teachers had some trouble 



Percentage of teachers 

Teachers’ Ratings 

Content 
(n = 21) 

Way Teach 
(n = 21) 

A great deal 

76.2 

81.0 

Moderately 

19.0 

14.3 

Somewhat 

4.8 

4.8 


Table 3. Teachers’ ratings of program’s influence on course content and way teach 
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Survey Administration 

Mean 

Standard deviation 

Pre Summer Program 

3.43 

1.12 

End of Summer Program 

4.71 

0.46 

* Wilcoxon signed ranks with Z = 

3.47 Sig. = .001, N = 

21 


Table 4. Mean Ratings of Competence in the Engineering Design Process* 


fitting a bioengineering design project into their curriculum. Some teachers obtained permission to 
provide an afterschool program or to create a new course when they could not fit the topic within 
their curriculum restraints. In this way, administrators also became clients for the teachers. 

Survey data also showed that teachers generally considered that they gained competence in the 
engineering design process. Teachers were asked at the beginning and end of the summer to rate “how 
competent you feel in the engineering design process.” Wilcoxon signed ranks tests of related samples were 
used to compare the teachers’ numerical responses before and after the program. Rating their competence 
on a scale from 1 “poor” to 5 “excellent,” teachers indicated statistically significant gains (Table 4). 

Teachers also “strongly agreed” (86.7%) or “agreed” (13.3%) that the program helped them to 
learn the EDP. 

Teachers also reported increases in their ratings of competence to “convey excitement about 
bioengineering to my students.” Their ratings went from between “3 -average” and “4 -good” at 
the start of the program to close to “excellent” at the end of the program (Table 5). The extent to 
which their higher ratings were due to the research projects in which they participated and/or the 
development and teaching of their curricula is unknown. 

Repeating teachers completed surveys at the beginning and end of each summer program in 
which they participated. The findings described above reflect their completion of surveys each 
summer. The first and last survey responses of the 15 teachers who participated in the program 
are plotted in Figure 4 (for the repeating teachers, only the results from the first pre and last post 
program surveys that were completed are plotted). 


Survey Administration 

Mean 

Standard deviation 

Pre Summer Program 

3.45 

1.30 

End of Summer Program 

4.86 

0.36 


*Wilcoxon signed ranks, with Z = 3.59 Sig.= .000, N = 21 


Table 5. Mean ratings of competence in conveying excitement about bioengineering to 
students* 
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Competence in the EDP Process 


Poor 

■ Fair 

■ Average 

■ Good 

■ Excellent 


o 

<u 

OD 



Competence in converying 
excitement about bioengineering 


Figure 4. Comparison of 15 teachers’ ratings of (a) competence in the Engineering Design 
Process and (b) competence in conveying excitement about engineering to their students. 
a Four teachers repeated the program once, one teacher repeated the program twice. 


A repeat teacher talked about the value of learning engineering design. His comments also reflect 
that it may take time for teachers to understand the richness of biomedical engineering and the EDP. 
At the end of the program, the teacher wrote, “I have been able to learn more about, understand and 
embrace the concept of design, which, three summers ago, had more of a superfluous meaning to me 
than it does now. I feel confident that / can use Biomedical Engineering in my teaching but the greatest 
benefit to me was learning about something that has importance in whatever project I do with students. 
That is design." 


ASSESSMENT OF STUDENTS 


Implementation and reporting of the planned student assessments was spotty. Few teachers 
talked about or reported on assessment, although all teachers created assessment plans for their 
lessons or units. A few teachers noted that they did not have time to implement their planned as¬ 
sessments. Others who ended up teaching the EDP in after school or other venues did not feel that 
assessment was appropriate. Informally, at follow-up sessions, the teachers reported that their stu¬ 
dents “loved” or “enjoyed” the lessons. They described students who “could not wait” to construct 
and test prototypes. 
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Because of funding limitations, we were able to hold follow-up meetings during the academic 
year for only two of the three years of the grant. During these meetings, teachers presented the 
implementation of their curriculum to their fellow participants. Participants offered suggestions 
for changes and noted positive aspects. While teachers’ presentations were formal, the discussion 
afterwards was informal. A few repeating teachers remarked that they made changes based on 
follow-up meeting comments. 

Reporting assessment results was recommended rather than required for this group of teachers. 
Documenting procedures and reporting results from pre and post student assessments are required 
for the middle school teacher participants in the new and following three-year grant. Again, teach¬ 
ers have latitude in determining appropriate assessments for their units and multiple measures and 
approaches are suggested. A more structured template for documentation and reporting is now 
provided. 


CONCLUSIONS AND SUMMARY 


Developing STEM lessons that address students’ needs and provide developmentally and peda- 
gogically appropriate content within classroom constraints is a continuous challenge for K-12 
teachers. In this paper, we provide a systematic, descriptive process for curriculum development 
that provides tools for teachers to approach this difficult task. Recognizing the parallel between 
Instructional Design approaches and the Engineering Design Process, we demonstrate how the EDP 
can be used to enrich and formalize development of K-12 STEM curricula. 

We found that most middle school teachers with whom we worked considered the process to 
help both their ability to design engaging lessons and their understanding of the EDP itself. By pro¬ 
viding structure, requiring the development of multiple alternative solutions, and calling attention 
to classroom constraints, the EDP process appears to be an effective process for guiding teachers 
toward more interactive and engaging project-based lessons. With the implementation of the Next 
Generation Science Standards many science teachers may find it easier, and more important, to 
include engineering design in their curriculum because the new standards stress the integration of 
the scientific and engineering fields (The Next Generation Science Standards 2012). 

Although in-depth laboratory research experience is the main focus of RET funding, the EDP 
process for curriculum development continues as an inherent part of our program. In the future, we 
plan to determine the extent to which our teachers continue to use the EDP after participation in 
the program and to evaluate the effect of the lesson plans on student learning and attitudes. The 
activities developed through this process will be disseminated through our website and compilation 
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sites such as TeachEngineering.org. We also plan to support additional teachers in their use of the 
EDP for creating new lessons, and we hope that dissemination of this work will broaden the use of 
the EDP for curricular development. 
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